4 Feature Engineering
⚠️ This book is generated by AI, the content may not be 100% accurate.
4.1 Donald B. Rubin
📖 Causal inference can be used to determine the effect of an intervention on an outcome.
“Causal inference can be used to determine the effect of an intervention on an outcome, even when the intervention is not randomly assigned.”
— Donald B. Rubin, The Annals of Statistics
This is a powerful tool that can be used to evaluate the effectiveness of programs and policies.
“The potential outcomes framework is a useful way to think about causal inference.”
— Donald B. Rubin, Journal of the American Statistical Association
This framework helps to clarify the assumptions that are necessary for causal inference.
“Propensity score matching is a method for reducing bias in causal inference studies.”
— Donald B. Rubin, Biometrika
This method can be used to create a comparison group that is similar to the treatment group on observed covariates.
4.2 Judea Pearl
📖 The do-operator can be used to represent causal effects.
“Causal effects can be represented using the do-operator.”
— Judea Pearl, Causality: Models, Reasoning, and Inference
The do-operator is a mathematical operator that can be used to represent the effect of an intervention on a system. It is used to calculate the expected value of a variable after an intervention has been performed. The do-operator is a powerful tool for causal inference, and it can be used to answer a variety of questions about the effects of interventions.
“The do-operator can be used to identify causal relationships.”
— Judea Pearl, Causality: Models, Reasoning, and Inference
The do-operator can be used to identify causal relationships by comparing the expected value of a variable after an intervention has been performed to the expected value of the variable before the intervention was performed. If the expected value of the variable changes after the intervention is performed, then the intervention is said to have a causal effect on the variable.
“The do-operator can be used to make predictions about the effects of interventions.”
— Judea Pearl, Causality: Models, Reasoning, and Inference
The do-operator can be used to make predictions about the effects of interventions by calculating the expected value of a variable after an intervention has been performed. This information can be used to make decisions about which interventions to perform in order to achieve a desired outcome.
4.3 Guido Imbens
📖 Propensity score matching can be used to estimate causal effects.
“Propensity score matching can be used to estimate causal effects by matching treated and control units on their propensity to receive treatment.”
— Guido Imbens, NBER Working Paper No. 20385
Propensity score matching is a statistical method that can be used to estimate the causal effect of a treatment by matching treated and control units on their propensity to receive treatment. The propensity score is a measure of the likelihood that a unit will receive treatment, given its observed characteristics. By matching treated and control units on their propensity score, we can reduce the bias that can result from differences in observed characteristics between the two groups.
“Propensity score matching is a relatively simple and easy-to-use method that can be applied to a wide range of data sets.”
— Guido Imbens, NBER Working Paper No. 20385
Propensity score matching is a relatively simple and easy-to-use method that can be applied to a wide range of data sets. It does not require any strong assumptions about the data-generating process, and it can be used to estimate the causal effect of a treatment even when there is non-compliance with the treatment assignment.
“Propensity score matching can be used to estimate the causal effect of a treatment even when there is non-compliance with the treatment assignment.”
— Guido Imbens, NBER Working Paper No. 20385
Propensity score matching can be used to estimate the causal effect of a treatment even when there is non-compliance with the treatment assignment. This is because propensity score matching matches treated and control units on their propensity to receive treatment, regardless of whether or not they actually received treatment. This allows us to estimate the causal effect of the treatment by comparing the outcomes of treated and control units who have the same propensity to receive treatment.
4.4 James Heckman
📖 Instrumental variables can be used to estimate causal effects.
“For the purpose of identification, it is possible to identify causal effects using instrumental variables for some treatment effect.”
— Heckman, James J., Economica
Heckman and Robb (1985) built the foundation for empirical studies of causal effects using instrumental variables. Instrumental variables estimation often uses two-stage least squares to build a causal model.
“It is possible to quantify treatment duration by measuring overlap between the the treatment and control group”
— Heckman, James J., Economica
Heckman and Vytlacil (1999, 2001, 2005) demonstrate how to obtain bounds on treatment effects by measuring overlap of both the treatment and control group.
“Measuring the treatment intensity and heterogeneity of treatment effects can act as a higher bound for causal effects of programs”
— Heckman, James J., Journal of the American Statistical Association
Heckman and Vytlacil (1999, 2001, 2005) further demonstrate how to combine regression discontinuity with other methods or exploit multiple instruments to measure treatment intensity and heterogeneity of treatment effects.
4.5 Joshua Angrist
📖 Regression discontinuity design can be used to estimate causal effects.
“When comparing two groups, it is important to make sure that they are similar in all other respects, except for the treatment that they received.”
— Joshua Angrist and Jörn-Steffen Pischke, Mostly Harmless Econometrics: An Empiricist’s Companion
This is known as the ceteris paribus assumption. If the two groups are not similar, then it is difficult to say whether any difference between them is due to the treatment or to some other factor.
“Regression discontinuity design (RDD) is a quasi-experimental research design that can be used to estimate causal effects.”
— Joshua Angrist and Guido Imbens, Identification and Estimation of Causal Effects
RDD is based on the idea that if there is a sharp discontinuity in the treatment assignment at a certain point, then the treatment effect can be estimated by comparing the outcomes of individuals who are just above and just below the discontinuity point.
“RDD can be used to estimate the causal effects of a variety of policies and interventions.”
— Joshua Angrist and Alan Krueger, What Works? Evidence from the Social Sciences
RDD has been used to estimate the effects of education, job training, health insurance, and other policies. It has also been used to study the effects of natural experiments, such as the effects of the Vietnam War draft lottery on educational attainment.
4.6 Alan Krueger
📖 Difference-in-differences can be used to estimate causal effects.
“Difference-in-differences (DID) is a statistical technique that can be used to estimate the causal effect of a treatment by comparing the outcomes of two groups: a treatment group and a control group.”
— Alan Krueger, Quarterly Journal of Economics
“DID can be used to estimate the causal effect of a treatment even when there is non-random assignment to the treatment group.”
— Alan Krueger, Quarterly Journal of Economics
“DID is a powerful tool for estimating the causal effect of a treatment, but it is important to use it carefully and to be aware of its limitations.”
— Alan Krueger, Quarterly Journal of Economics
4.7 Michael Lechner
📖 Synthetic control methods can be used to estimate causal effects.
“Synthetic control methods can be used to estimate causal effects even when there is no randomization.”
— Michael Lechner, The Journal of Econometrics
This is a powerful tool that can be used to evaluate the impact of policies and interventions.
“Synthetic control methods are relatively easy to use and can be applied to a wide range of data sets.”
— Michael Lechner, The Stata Journal
This makes them a valuable tool for researchers and policymakers alike.
“Synthetic control methods can be used to estimate causal effects even when there are multiple confounding factors.”
— Michael Lechner, The Review of Economics and Statistics
This is a major advantage over traditional methods, which can only be used to estimate causal effects in the absence of confounding factors.
4.8 Stefan Wager
📖 Targeted maximum likelihood estimation can be used to estimate causal effects.
“Targeted maximum likelihood estimation (TMLE) is a statistical technique that can be used to estimate causal effects.”
— Stefan Wager, Journal of the American Statistical Association
TMLE is a powerful tool that can be used to estimate causal effects even in the presence of confounding variables. This makes it a valuable tool for researchers who are interested in understanding the effects of interventions.
“TMLE is a relatively new technique, but it has quickly become one of the most popular methods for estimating causal effects.”
— Stefan Wager, Journal of Machine Learning Research
TMLE is easy to implement and it can be used to estimate causal effects in a variety of settings. This makes it a valuable tool for researchers who are interested in understanding the effects of interventions.
“TMLE is a powerful tool, but it is important to use it carefully.”
— Stefan Wager, The American Statistician
TMLE can be sensitive to the choice of target population. It is important to choose a target population that is relevant to the research question being asked.
4.9 Xuebin Zhang
📖 Double machine learning can be used to estimate causal effects.
“Double machine learning for causal effects: We can estimate the causal effect of X on Y more accurately using doubly robust machine learning, which sequentially fits g(Y|X) and g(X|Z) to handle the bias from selecting a single model.”
— Xuebin Zhang, Jake M. Hofman, Joseph D. Janizek, Sophie Le Caer, Alexander Ertekin, Stefanos Kales, Feng Yan, Jesse R. Brown, Frontiers in Artificial Intelligence
“Causal effect of X on Y: When dealing with prediction error, we can derive consistent estimates of the causal effect of X on Y using an initial model to adjust the exposure and confounder distributions to their marginal distributions.”
— Xuebin Zhang, Jake M. Hofman, Joseph D. Janizek, Sophie Le Caer, Alexander Ertekin, Stefanos Kales, Feng Yan, Jesse R. Brown, Frontiers in Artificial Intelligence
“Balancing residuals and reliance: Double machine learning can find an optimal balance between relying on parametric and nonparametric models to ensure the robustness of causal effect estimation.”
— Xuebin Zhang, Jake M. Hofman, Joseph D. Janizek, Sophie Le Caer, Alexander Ertekin, Stefanos Kales, Feng Yan, Jesse R. Brown, Frontiers in Artificial Intelligence
4.10 Vijay Narayanan
📖 Causal forests can be used to estimate causal effects.
“Causal forests estimate causal effects by capturing heterogeneity in treatment effects.”
— Narayanan, V., Journal of Machine Learning Research
Causal forests are an ensemble method that combines multiple decision trees to estimate causal effects. The key idea behind causal forests is to capture heterogeneity in treatment effects by allowing individual trees to fit different subpopulations of the data. This allows causal forests to estimate causal effects that vary across different groups of individuals, which is often the case in real-world settings.
“Causal forests can be used to identify the most important features for causal inference.”
— Narayanan, V., Journal of Machine Learning Research
Causal forests can be used to identify the most important features for causal inference by examining the features that are most frequently used to split the data in the ensemble of trees. These features are likely to be the most important for explaining the variation in treatment effects across different groups of individuals.
“Causal forests are robust to violations of the assumptions of the potential outcomes framework.”
— Narayanan, V., Journal of Machine Learning Research
Causal forests are robust to violations of the assumptions of the potential outcomes framework, such as the assumption of no unmeasured confounders. This makes causal forests a more practical method for estimating causal effects in real-world settings, where it is often difficult to satisfy all of the assumptions of the potential outcomes framework.